LIBRARY 

OF  THE 

MASSACHUSETTS  INSTITUTE 
OF  TECHNOLOGY 


NOTES  ON  MARKETING  EXPERIMENTATION 
99-64 
John  D."  C.  Little 


MASSACHUS] 

TITUTE  OF  TECHNOLOGY 
50  MEMORIAL  DRIVE 


NOTES  ON  MARKETING  EXPERIMENTATION 
99-64 
John  D.  C.  Little 


October  15,  196A 

Sloan  School  of  Management 


* 

Notes  on  Marketing  Experimentation 


1.0  Introduction 


The  successful  conduct  of  marketing  operations  requires  good  information 
about  market  behavior.   Such  information  comes  from  many  sources — from  his- 
torical sales  data^  personal  experiences^  surveys,  and  various  special  studies. 
We  shall  here  take  up  a  specialized  but  valuable  method  of  learning  about  the 
market,  namely,  experimentation. 

Principally,  we  shall  consider  experiments  involving  the  active  inter- 
vention in  the  market  by  a  company  to  measure  the  effect  on  sales  of  variables 
under  the  company's  control.   For  example,  quantities  such  as  advertising, 
price,  salesman's  calls,  packaging,  and  display  might  be  varied  to  estimate 
their  effect  on  sales.   Presumably,  once  the  sales  effect  is  estimated, 
appropriate  calculations  will  convert  this  into  an  effect  on  profit  or  other 
relevant  measure  of  effectiveness.   Although  the  discussion  will  focus  on  sales 
experiments,  most  of  the  principles  discussed  apply  to  any  response  variable. 

The  term,  experiment,  will  be  used  to  imply  a  controlled  comparison  of 
alternatives.   In  other  words,  two  or  more  experimental  treatments  are  applied 
in  a  situation  where  the  experimenter  is  able  to  decide  which  experimental 
units  receive  the  treatment.   Thus,  an  experiment  might  compare  promotions 
A  and  B  by  using  them  in  different  sets  of  cities.   The  experimenter  decides 
which  city  receives  which  treatment  (probably  doing  this  by  a  random  process). 
A  more  passive  approach  is  to  analyze  historical  data  in  which  marketing 
variables  have  varied  in  the  normal  course  of  company  operations.   Such  data  can 
sometimes  be  analyzed  by  econometric  methods  to  estimate  the  effect  of  the 
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variables  on  sales.   For  example,  Telser  has  used  such  methods  to  estimate 
the  effect  of  advertising  on  cigarette  sales     and  price  on  the  demand  for 
certain  branded  goods.'-  J  Historical  data  may  be  quite  revealing  in  the 
situation  that  Howard  and  Roberts  have  called  a  try-out.*-     A  company  may 
try  out  a  new  policy  but  without  benefit  of  a  control  or  alternative  treatment. 
If  the  change  is  drastic  enough,  say  a  price  cut  or  the  introduction  of  a  new 
product,  there  are  sometimes  special  opportunities  for  analysis.  An  example 
of  this  is  Henderson  and  Brown's  study  of  the  effect  of  a  promotional  campaign 
on  the  sales  of  frozen  orange  juice. 

Although  the  analysis  of  historical  data  is  often  very  valuable,  experiments 
offer  several  special  advantages.   If  you  make  an  experimental  change  and  observe 
an  effect,  it  is  frequently  more  convincing  than  if  a  change  and  an  effect  occur 
together  in  past  data.   In  the  latter  case,  you  are  likely  to  worry  that  a 
deeper  mechanism  may  be  causing  both  the  change  and  the  effect,  s  ince  changes 
in  marketing  variables  are  usually  made  for  some  reason.   A  serious  difficulty 
in  econometric  analyses  is  that  the  variables  of  interest  may  have  substantial 
correlations  with  each  other  and  with  other  explanatory  variables.  This  may 
lead  to  instability  in  estimates  of  the  important  regression  coefficients.  In 
designing  an  experiment  a  deliberate  effort  is  made  to  have  small  correlation 
among  experimental  variables  and  between  them  and  other  explanatory  variables. 
Finally,  of  course,  experiments  can  be  directed  at  questions  of  immediate 
interest  and  can  examine  more  different  alternatives  than  would  normally  occur 
in  regular  operations. 

Experimentation  has  some  important  limitations.   These  will  become 
clearer  a  little  later.   Principally,  however,  the  accuracies  obtainable,  the 
time  required  for  performance,  and  in  some  cases,  the  cost  of  the  study 
clearly  limit  the  number  of  questions  which  can  be  turned  over  to  strict 
empiricism  of  this  sort. 
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In  any  case^  sales  experiments  are  best  supported  by  studies  of  Inter- 
mediate variables  (eg.  consumer  attitude  and  awareness  shifts,  and  reactions 
In  the  distribution  system)  that  may  Increase  understanding  of  marketing 
processes  and  may  eventually  permit  good  predictions  of  sales  effectiveness 
on  the  basis  of  Intermediate  variables. 

An  Interesting  potential  application  of  sales  experiments  Is  in  the  role 
of  a  nionltoring  device  In  the  continuous  control  of  marketing  operations.  At 
least  one  company  does  this  in  an  Informal  way  now.  In  certain  fields^ 
notably  chemical  engineering,  such  control  is  well  known  under  the  title  of 
evolutionary  operations.  Figure  1  traces  out  the  main  ideas  in  marketing 
terms.  Small  experiments  are  made  a  part  of  normal  marketing  operations. 
Information  from  the  experiments  is  fed  back  into  the  budget  allocation 
process.  Although  to  our  knowledge  no  satisfactory  fortttal  theory  presently 
exists  for  the  marketing  application,  the  prospect  is  sufficiently  interesting 
that  we  shall  discuss  some  of  the  problems  Involved  later  on. 

Many  books  have  been  written  on  the  statistical  design  of  experiments. 
Cox''-'  gives  a  reasonably  non-technical  exposition  of  the  basic  ideas. 
Cochrane  and  Cox'--'  is  something  of  a  handbook.  Kempthorne  I- J  goes  deeper 
into  the  theory. 

2.0  Example;  A  Two  Level  Spending  Experiment 
As  a  starting  example,  consider  the  problem  of  measuring  sales  response 
to  promotional  spending  in  order  to  find  a  more  profitable  spending  rate. 

2.1  Profit  Model 

We  suppose  that,  under  some  given  set  of  conditions,  sales  rate  Increases 
with  promotional  spending,  fairly  rapidly  at  low  spending  levels  but  with 
diminishing  returns  at  high  levels.  The  sales  response  curve  might  look  some- 
thing as  follows: 


-  4  - 


Experimental 
perturbation 
of 
allocation 


i' 


Determination 
of  optimal 
allocation 


Implementation 

of 

allocation 


< 


Estimation 
of  sales 
response 
functions 


Consumer 
population 


S 


Distribution 
system 


^ 


Sales 
rate 


\y 


Profit 
rate 


Figure  1.   Block  diagram  for  using  continuous  experiments  in  the  control  of 
marketing  effort. 
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Figure  2 


Here    s  =  sales  rate,  say,  in  units  of  dollars  per  household  per  year 
(dol/hh.yr) 

X  =  promotion  rate  (dol/hh.yr) 

(More  obvious  units  for  sales  and  promotion  might  be  dollars/year  but,  if 

different  sized  regions  are  to  be  compared,  some  division  by  a  size  variable 

is  desirable.   A  good  divisor  is  a  quantity  that  is  a  measure  of  potential 

sales  in  the  region.   Although  individual  products  frequently  have  their  own 

special  potentials,  population  and  number  of  households  are  good  all-round 

measures  of  size.  The  latter  will  be  adopted  here.) 

To  calculate  the  value  of  x  which  maximizes  profit  rate,  we  construct 

a  simple  model  of  company  profit.   Let 

p  =  profit  rate  (dol/hh.yr.) 

m  =  gross  margin,  i.e.,  incremental  profit  as  a  fraction  of  sales 
(dimensionless) 


Then 


selli 


ns_2. 


rice  -  variable  cost 


selling  price 
c  =  fixed  cost  rate,  exclusive  of  the  promotional  spending  (dol/hh.yr) 


p=ms-x-c  (2.1) 

We  are  presuming  that  the  type  of  promotion  being  investigated  is  a  fixed  cost 
(eg.  advertising)  not  a  variable  cost  (eg.  a  .price  deal).  A  somewhat  different 
model  is  appropriate  for  the  latter  case. 


Given  the  sales  response  function  s  =  s(x),  the  operating  point  for 
maximum  profit  can  be  determined.  Thus,  by  calculus, 

0 


— c  =  m 1 

dx     dx 


or,  the  best  value  of  x  is  the  one  that  makes 

ds  ^  J, 
dx   m 


(2.2) 


Suppose  that  ra  =  33-1/3%.  Then  1/m  =  3.  For  a  curve  of  the  general  shape  of 
Figure  2,  the  maximum  profit  value  of  x  can  be  found  by  increasing  x  until 
the  slope  of  the  curve  is  3,  i.e.,  until  an  additional  dollar  spent  returns 
3  dollars  in  sales. 

Much  more  complicated  profit  models  can  be  constructed.   If  budgets  are 
limited  and  good  alternative  uses  exist  for  the  money,  higher  slopes  than  1/m 
may  be  required  for  maximum  profitability.  Time  lags  could  be  introduced  and 
future  profits  discounted. 

The  simple  model  suffices,  however,  to  motivate  an  experiment  for 
estimating  sales  response  curve  and  to  provide  a  reference  point  for  judging 
how  accurate  the  experiment  must  be  in  order  to  be  meaningful.  An  experiment 
with  a  standard  error  for  the  slope  of,  say,  10  when  1/m  =  3  would  probably 
be  useless.   A  requirement  that  an  experiment  have  a  standard  error  of  .01 
might  lead  to  unnecessary  expense  or  perhaps  an  impossibility. 

2.2  An  Experimental  Setup 
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Figure  3 


Each  point  is  a  city 


spending  (dol/hh.yr) 


Suppose  we  pick  one  set  of  cities  and  apply  a  low  spending  rate  and 
another  set  and  apply  a  high  spending  rate.   The  spending  rates  are  used  for 
a  test  period  of^  say,  a  year  and  the  resulting  sales  are  observed. 

Let    x^  =  low  spending  rate  (dol/hh.yr.) 

x^  =  high  spending  rate  (dol/hh.yr.) 

n^  =  number  of  cities  in  low  group 

n_  =  number  of  cities  in  high  group 

s..  =  sales  rate  in  the  i —  city  of  low  group  during  the  test 
period  (dol/hh.yr.) 

s  =  —  Zs  .  =  average  sales  rate  in  low  group  (dol/hh.yr.) 

1  n^   li 

S   =  sales  rate  in  i —  city  of  high  group  (dol/hh.yr.) 

s-  =  —  Es^  =  average  sales  rate  in  high  group  (dol/hh.yr.) 

2  n-   2i 

Denote  the  slope  by  b  and  its  estimate  by  b.   A  straightforward 
estimate  of  b  is: 

.    ^2  ■  ^ 


(2.3) 


The  particular  slope  estimated  is  that  of  the  chord  of  the  sales  response 
between  x  and  x  •   (Strictly  speaking,  the  experiment  can  only  tell  us  which 
of  the  two  points  x.  and  x  is  more  profitable.   However,  if  the  curve  is 
smooth  between  the  two  values,  it  will  take  on  the  slope  of  the  chord  some- 


than  two  spending  rates,  fit  a  curve  through  them  and  obtain  a  continuously 
varying  slope  over  a  considerable  range  of  x.) 
2.3  The  Statistical  Model 

A  reasonable  statistical  model  to  go  with  the  above  setup  is  as  follows. 
Assume  the  observed  sales  rate  in  a  city  having  promotion  rate  x  is  a  random 


variable^  s  : 

s  =  s(x)  +  £ 

where  s(x)  is  the  sales  response  curve  and  £  is  a  random  variable  of  mean  zero 

2 
and  variance  a     .     The  variable  6  is  assumed  independent  from  city  to  city 

and^  for  concreteness^  will  be  assumed  to  be  normally  distributed. 

In  other  words^  we  assume  that  there  is  an  underlying  average  sales 

response  curve  in  the  class  of  cities  under  study  and  that  the  cities  in  each 

test  group  will  be  scattered  about  the  curve  with  a  standard  deviation  of  g 

at  their  particular  x  value. 

2.3  The  Standard  Error  of  the  Estimated  Slope 

The  following  notation  will  be  used.   Let  X  and  Y  be  random  variables. 
E(X)  =  expected  value  of  X 
V(X)  =  variance  of  X 
SE(X)  =  [y(X)J  2  =  standard  error  of  X 
CV(X)  =  SE(X)/E(X)  =  coefficient  of  variation  of  X 
Cov  (X,Y)  =  covariance  of  X  and  Y 
Let  a  and  b  be  constants.   A  useful^  standard  formula  is: 

V(aX  +  bY)  =  a^V(X)  +  b^V(Y)  +  2ab  Cov(X,Y)  (2.5) 

The  estimate  of  slope  previously  given  is 


This  is,  in  fact,  the  maximum  likelihood  estimate  under  the  assumed  statistical 
model . 

We  want  SE(b).  Using  (2.5) 

V(b)  = J   V(7,  -  s,)  . 

^-2-V     '    ' 

Because  of  the  independence  of  the  individual^  's,  Cov  is  ,'s,)   =   0  and  so 

VG2  -   s^)  =  vd^)  +  V(7^) 


9  - 


_      2     _      2 

V(s-  )  =  -H-  ,  v(sJ  =-^ 
1    n   '     2    n„ 


Putting  things  together;  1. 

2 


SB(b)  -,-^  -  +  -T 
^V^^L^l    "2J 


(2.6) 


This  formula  tells  the  accuracy  of  the  experiment.   Suppose  for  example 

0   =  .02  (dol/hh.yr.) 

X  =  .01  (dol/hh.yr.) 
n,  =  10  ^ 

^  X  =  .02  (dol/hh.yr.) 

n2  =  10 

Then 

SE(b)  =  .9 

If^  say^  1/m  =  3,    we  see  that  we  can  easily  tell  the  very  good  from  the  very 

bad,  but  for  b  close  to  1/m  it  will  be  hard  to  tell  whether  we  are  over  or 

under  the  best  value  of  x.   Of  course,  if  the  operation  is  fairly  close  to  the 

optimum,  we  may  be  reasonably  well  satisfied,   because  most  meixima  are  smooth 

near  the  top  and  small  deviations  may  not  make  much  difference. 

The  formula  also  shows  what  quantities  affect  accuracy.   Increasing  the 

sample  sizes  will  decrease  the  standard  error  according  to  the  familiar  square 

root  rule.   Any  decrease  that  can  be  made  in  a  decreases  the  standard  error 

proportionately.   The  quantity  (x  -x^)  enters  the  expression  in  an  extremely 

sensitive  way.   Small  values  will  give  very  high  standard  errors.   The  reason 

is  simple:   in  order  to  detect  a  sales  difference  amid  the  random  variations, 

one  must  create  a  fairly  substantial  sales  difference  and  this  will  usually 

require  fairly  well  separated  promotional  rates.   On  the  other  hand,  although 

large  values  of  x„-x^  are  desirable,  caution  is  necessary  because  too  large 

:omot: 

make  the  slope  of  the  chord  between  x,  and  x  look  small,  even  though 
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An  alternative  form  of  (2.6)  is  sometimes  convenient: 

1 


The  values  of  s  and  x  can  be  anything  since  they  cancel  out.   Let  us  suppose 
they  are  the  national  averages  of  the  sales  rate  and  the  promotion  rate 
expressed  In  dol/hh.yr.  Then  all  the  terms  in  parentheses  are  dimensionless 
and  have  the  following  interpretations:   a/s  is  the  coefficient  of 
variation  of  sales  for  cities  of  the  type  used  in  the  experiment,  s/x  is  the 
ratio  of  sales  to  promotion  nationally;  and  x/(x  -  x  )  is  the  ratio  of  the 
national  promotion  rate  to  the  difference  between  the  two  test  rates. 

A 

It  is  important  to  estimate  SE(b)  before  doing  an  experiment  like  the 
above.   Only  then  can  one  decide  whether  the  experiment  is  worth  doing.   What 
we  principally  need  to  know  for  this  purpose  is  a.      In  the  next  section  we 
shall  present  the  details  of  a  two  level  spending  experiment  and  its  analysis. 
In  the  course  of  the  analysis  we  shall  see  one  way  that  o  can  be  estimated 
in  advance. 

2.4  Numerical  Example 

Next  we  present  and  analyze  data  adapted  from  a  two  level  advertising 
experiment  conducted  a  few  years  ago  by  the  manufacturer  of  a  grocery  Store 
product. 

The  objective  of  the  experiment  was  to  measure  the  effect  of  spending 
rate  on  sales.   The  experimental  units  were  the  company's  sales  territories. 
Out  of  a  national  total  of  21  territories,  18  were  given  the  normal  advertising 
rate,  while  3  were  given  twice  the  normal  rate.   The  test  ran  for  a  year. 
Sales  were  measured  by  factory  shipments. 

To  portray  the  statistical  characteristics  of  the  original  ;data  but- still 
conceal  the  actual  numbers  and  their  ratios,  we  have  taken  the  following  steps: 
Sales,  advertising,  and  the  measure  of  potential  have  all  been  multiplied  by 
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scale  factors  to  standardize  them  at  national  values  of  approximately  10  million 
dollars/year  for  sales^  .5  million  dollars/year  for  advertising  and  50  million 
households  for  potential.   The  data  in  this  form  have  been  analyzed  to 
estimate  the  parameters  of  the  statistical  model.   Thereafter  a  set  of  new 
data  has  been  generated  by  simulating  the  statistical  model.   It  is  the  simu- 
lated data  that  is  tabulated  here. 

Table  I  presents,  for  each  territory,  sales  during  the  test  year,  sales 
the  previous  year,  advertising  during  the  test  year,  and  the  measure  of 
potential.   Table  II  puts  the  sales  and  advertising  data  on  a  per  household 
basis.   The  high  spending  rate  territories  are  clearly  1,  2,  and  3. 

Two  analyses  will  be  given,  the  second  being  neater  than  the  first. 
The  first  is  presented  because  it  is  the  basic  intuitive  analysis.  The 
second  represents  a  worthwhile  embellishment. 

Analysis  I:   The  average  values  of  sales  and  advertising  in  the  high 
and  low  groups  are: 

7  =  .24796  dol/hh.yr.  x^  =  .01675  dol/hh.yr. 

I  =  .21300     "  x^  =  .00892 

and  so  an  estimate  of  the  slope  of  the  sales  response  curve  is: 


Thus,  it  is  estimated  that  in  increasing  the  spending  rate  from  x  to  x  , 
each  dollar  of  advertising  brough  back  4.46  dollars  of  sales  during  the 

experimental  period. 

A 

Next  we  want  SE(b)  and  for  this  purpose  an  estimate  of  a,    the  standard 

deviation  of  the  dependent  variable  in  the  population  of  experimental  units. 
Were  it  not  for  the  presence  of  the  experimental  treatment,  we  could  simple  find 
the  standard  deviation  of  the  number  in  the  first  column  of  Table  II.   As  it  is, 
slightly  different  means  should  be  subtracted  from  the  high  and  low  groups. 
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CXPERIMENT    U   TABLE    I 


SALES  IN 

ADVERTISING  IN 

SALES  IN 

TCRRIT0RV 

TEST  YEAR 

TEST  YEAR 

PREVIOUS  YEAR 

POTENTIAL 

MO. 

fO0LLARS> 

(DOLLARS) 

(DOLLARS) 

(HOUSEHOLDS) 

I 

$86157 

45030 

921912 

2654913 

2 

612242 

37950 

424086 

2252610 

» 

955606 

36393 

490691 

2211568 

4 

1505579 

63565 

1302596 

7645118 

S 

373427 

16664 

296753 

2169792 

1 

403184 

18623 

394744 

1979986 

515501 

22763 

470941 

2721930 

JB 

759243 

25743 

704022 

2917827 

^ 

414064 

14926 

429551 

1696091 

10 

390888 

14941 

310568 

1696091 

11 

284007 

11998 

250089 

1376431 

12 

993965 

26892 

997926 

2634292 

13 

395453 

15936 

333822 

1819779 

14 

404030 

16301 

393962 

1792798 

1» 

276128 

12873 

291158 

1464068 

16 

391713 

20151 

373694 

2009361 

17 

422276 

17283 

469825 

1912968 

18 

726697 

30097 

695916 

3283844 

19 

922147 

22328 

531H4 

2453861 

20 

369997 

15379 

386076 

1747603 

21 

365622 

12564 

368556 

1587793 

TOTALS 

10626048 

900000 

10000000 

90000000 
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EXPERIMENT  1*  TABLE  II 


S(T) 
TERRIT0RY    SALES  IN 
N0*     (O0L/HH,YR*I 


XJTI  SIT-1» 

ADVERTISING  IN    SALES  IN 
(O0L/HH«YR*)     IO0L/HH«YR«> 


.22078 

.01696 

•19658 

•27176 

•01684 

•18824 

.25131 

•01645 

•22187 

.19693 

•00831 

•17038 

.17084 

•00853 

•13576 

.20367 

•00940 

•19940 

.18938 

•00836 

•17301 

.26020 

•00882 

•24128 

•24413 

•00880 

•25326 

.23046 

•00880 

•18311 

.20633 

•00842 

•18169 

.22547 

•01020 

•21179 

.19532 

•00875 

•18344 

.23051 

•00930 

•22476 

.18860 

•00879 

•19886 

.19533 

•01004 

•18634 

.22079 

•00903 

•24565 

.22128 

•00916 

•21192 

.21278 

•00909 

•21644 

•21169 

•00880 

•22206 

.23027 

•00791 

•23211 
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21        -  2  .3 

Z   (s        -   s  )   =  8.32  X  10 
i=4 

^       -  -3 

Z  (S2i  -  ^2^  =  1.21  X  10 

i=l 

An  unbiased  estimate  of  a   is: 

I 
2 


[95.3 
=  [19-^ 


.0224 


Therefore,  the  estimated  standard  error  of  the  slope  is 
SE(b) 


P  _   /I  ,  i  1  2  =  .0224  fl  +  ij  =  1.78 
"  ''l   L"l   "2 J      .00783  |j8   3J 


/S  A 

To  summarize:        b  =  4.46  ,    SE(b)  =1.78 

Analysis  II.   If  further  variables  can  be  found  that  help  to  explain 

the  variation  in  sales  from  territory  to  territory,  the  standard  error  can 

be  reduced  and  greater  accuracy  obtained.   As  is  well  known,  sales  in  an  area 

tend  to  be  like  previous  sales  in  that  area.   Previous  sales  almost  always 

makes  a  good  explanatory  variable.   Let 

3  (t)  =  sales  rate  in  the  i —  territory  in  the  test  year  (dol/hh.yr.) 

s^(t-l)  =  sales  rate  in  the  i—  territory  in  the  preceding  year  (dol/hh.yr.) 

X  (t)  =  advertising  rate  in  the  i —  territories  in  the  test  year 
^      (dol/hh.yr.) 

^.  =  a  random  error  for  i —  territory  (dol/hh.yr.) 

A  simple  regression  model  for  sales  is  then 

s.(t)  =  a^  +  a^x^(t)  +  a2S^(t-l)  +^^  (2.8) 

where  the ^  are  assumed  independently  and  normally  distributed  with  mean 

2 
zero  and  variance  a   . 

The  model  is  linear  in  x  even  though  the  sales  response  curve  may  not  be. 

This  is  because  we  have  only  two  spending  levels  and  seek  the  slope  of  the  line 
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connecting  them.   XAlthough  x  values  are  not  exactly  alike  within  each  group, 
there  is  not  nearly  enough  variation  to  estimate  non-linearity.)   The 
regression  coefficient  a  corresponds  to  the  slope  b. 

A  regression  using  the  data  in  Table  II  in  the  Model  (2.8)  yields  (see 


Analysis  II  in  Table  III): 


A 

b  =  4.40 


SE(b)  =  1.25 


The  standard  error  of  b  has  dropped  about  30%  due  primarily  to  a  drop  in  a 
from  .0224  to  .0159  . 

Analysis  I  can  also  be  done  as  a  regression  by  dropping  s.(t-l)  as  a 
variable  in  (2.8).   The  results  are  shown  in  Table  III.   Analysis  II  is 
obviously  better  and  is  taken  as  the  final  set  of  results. 

Discussion.   What  can  we  say  about  these  results  from  a  managerial 
point  of  view?   Suppose  our  criterion  is  1/m  =  2.5.   We  can  say  the  higher 
level  looks  pretty  good,  although  there  is  always  a  chance  it  is  not  best. 
(b  and  2.5  are  separated  by  about  1.5  standard  errors.) 

On  the  other  hand,  suppose  1/m  =4.5  .The  estimate  does  not  differentiate 
between  x.  and  x  .   Possibly,  however,  a  manager  has  prior  feelings  on  adver- 
tising effectiveness.   These  can  be  taken  into  account  intuitively,  or  if 
made  explicit  as  prior  probabilities,  incorporated  in  a  Bayesian  analysis. 

The  experiment  has  yielded  information  but  uncertain  information.   Values 
far  away  from  4.40  are  now  considered  improbable.   If,  as  is  frequently  the 
case,  the  company  had  little  previous  knowledge  on  sales  response,  the 
increase  in  information  may  be  considerable. 

What  was  the  cost  of  the  experiment?  The  sales  data  came  from  factory 
shipments  and  would  have  been  collected  anyway. .  The  cost  of  the  analysis  was 
small.  The  money  spent  on  the  increased  advertising  was  an  immediate  out-of- 
pocket  cost,  which  may  or  may  not  have  been  recovered.  However,  if  1/m  <  4.40 
the  appearances  are  that  the  experiment  showed  a  net  profit. 
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EXPERIMENT  1*  TABLE  III 


MEANS 
SIT)  X(TI 

•21600        .01004 


S(T-n 
•20372 


STANDARD  OEVIATI0NS 
S(T»  XIT)         SfT-l» 

•  02<>6656      •0027899      ^0279975 

C0RRELATI0N  MATRIX 
S(T)  X<T»         SIT-l) 

S(T)     1.000000 
XJTI      .498762      1.000000 
S(T-l)     .627725       .001619      1*000000 


ANALYSIS  I 

MULT*  R  >   .4>967.    ST.  ERR.  0F  EST.  « 
F(l«19>  «  6.2916 


>0224 


VAR 


TABLE  0F  RESULTS 
B-C0EFF 


CONSTANT  .1736 
X(T)   4.4131 


.ERR.B 

T 

.0183 
1.7593 

2.5063 

NALYSIS  II 

MULT. 


8011*    ST.  ERR.  0F  EST.  - 
F(2, 18)  ■  16.1249 


.0159 


VAR 


TABLE   0F   RESULTS 
B-C0EFF         ST.ERR.a 


C0NSTANT  .0611 

*0284 

. 

XIT)   4*4041 

1.2482 

'       3*5283 

SIT-l)   *5527 

.1243 

4*4440 
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The  experimetit  might  be  Itnpuoved.   Perhaps  -a  better  statistical  model  and 
analysis  can  be  found.   More  experimental  units  in  the  high  group  would  help, 
although  this  would  mean  more  out-of-pocket  cost.   Possibly  a  change  of  experi- 
mental unit  from  sales  territory  to  metropolitan  markets  would  be  desirable. 

An  estimate  of  0  Is  vital  to  the  sound  planning  of  an  experiment. 
Analysis  II  can  be  adapted  to  do  this  by  dropping  x(t)  and  running  it  on 
historical  data.  The  value  of  a  is  not  likely  to  vary  much  from  year  to 
year.  Even  though  a  is  estimated  from  (2.8)  it  is  the  relevant  quantity 
for  (2.6)  in  a  two  level  spending  experiment.  We  may  think  of  the  sales 
data  in  (2.3)  as  being  adjusted  for  sales  in  the  previous  year. 

A  convenient  way  to  express  a   is  as  the  coefficient  of  variationj  the 
ratio  of  standard  deviation  to  the  mean.   From  Table  III: 


CV  =  .0159  =  7.3% 
.2180 

A  revealing  way  to  look  at  the  standard  error  of  the  difference  between 

treatments  ie   to  express  it  as  a  percent  of  sales.  Here 

2 


^2  -  ^1)  _  _!.  fi  -f  n 


=  .073 

=  .045 
Thus,  the  advertising  increase  must  produce  a  percentage  increase  in  sales 
which  will  show  up  against  a  4.37.  standard  error. 


3.0  Designing  Experiments 

3.1   Choice  of  dependent  variable 

By  "dependent  variable"  we  mean  the  numerical  quantity  that  the  experi- 
mental treatments  are  supposed  to  affect.   The  focus  here  is  on  sales^ 
although  a  variety  of  other  variables  are  frequently  of  interest  and  many 
of  the  remarks  made  here  apply  with  little  change  to  other  variables.   An 
important  dependent  variable  closely  related  to  sales  is  market  share. 

Market  share  usually  has  quite  a  bit  less  variance  than  sales  itself 
and  so  permits  a  more  sensitive  measurement.  Seasonal  effects  tend  to  be 
removed  as  do  some  of  the  effects  of  national  economic  conditions.  Hov;/ever^ 
if  a  company's  marketing  treatments  increase  competitive  sales  as  well  as 
company  sales^  the  percent  increase  in  market  share  will  be  less  than  the 
percent  increase  in  sales.  Whenever  possible,  competitive  sales  should  be 
examined  for  evidence  of  experimental  effects. 

Obtaining  satisfactory  sales  data  is  sometimes  difficult.   An  organi- 
zation working  directly  with  the  final  customer^  eg.  a  supermarket  or  a  mail 
order  house^  can  audit  its  own  sales.   A  manufacturer  can  use  factory  ship- 
ments, but  then  must  accept  delays  in  the  pipeline.   This  is  tolerable  if 
the  experiment  lasts  a  considerable  length  of  time.   If  the  product  is 
shipped  to  regional  warehouses  of  retail  chains,  some  difficulty  may  arise 
in  allocating  the  sales  to  individual  market  areas,  although  some  workable 
system  can  usually  be  found. 

Store  audits  are  an  effective  but  usually  costly  way  to  measure  sales. 
The  Department  of  Agriculture  experiments  have  usually  used  store  audits.. 
Consumer  surveys  are  another  device.   Both  surveys  and  audits  offer  the 
possibility  of  collecting  data  on  competitive  sales.   Some  products  have 
specialized  opportunites  for  collecting  sales  data  (eg.  warranty  cards). 


The  advantage  of  collecting  data  on  extra  dependent  and  independent 
variables  is  worth  emphasis.   The  advantages  are  illustrated  in  many  of  the 
Department  of  Agriculture  experiments  where  sales  of  related  products  were 
recorded^ as  were  prices^  display  space,  point-of-sale  promotions,  and  media 
advertising.   Frequently,  the  results  gave  information  on  these  added 
variables,  including,  from  time  to  time,  interesting  surprises. 

3. 2  Choice  of  experimental  unit 

The  term  "experimental  unit"  is  used  to  refer  to  the  unit  that  is 
assigned  a  treatment  independently  of  the  other  units.   Thus,  if  a  market 
area  is  given  a  "high  television"  treatment,  the  individuals  in  the  market 
may  be  considered  to  have  received  this  treatment,  but  the  experimental 
unit  is  the  market  not  the  individual.   This  is  because  each  individual 

is  not  (and  in  practice  could  not  be)  assigned  a  high,  low  or  other 
television  treatment  independently  of  the  others  in  the  market. 

The  most  desirable  experimental  unit  is  probably  the  individual,  or, 
as  is  frequently  more  appropriate,  the  household.   Depending,  of  course, 
on  the  situation,  sample  sizes  in  the  1000' s  or  100,000' s  may  be  possible. 
Although  coefficients  of  variation  for  individuals  are  greater  than  for 
more  aggregated  units,  the  reductions  in  standard  error  are  likely  to  be 
substantial.   Individuals  or  households  can  be  used  as  experimental  units 
in  direct  mail  promotion,  magazine  split  runs  and  personal  selling,  provided 
that  sales  can  be  traced  to  the  individuals.   Direct  mail  sales  experimenta- 
tion has  had  a  long  and  successful  history  in  such  applications. 

An  individual  store  can  be  an  experimental  unit  for  treatments  directed 
at  individual  stores.   For  advertising  treatments  involving  television, 
newspapers,  or  the  like,  the  experimental  unit  is  usually  a  market  area. 
In  some  of  the  Department  of  Agriculture  experiments  fil  ["2']   the  unit 
consisted  of  a  group  of  stores  within  the  market.   Some  companies  have  used 
their  sales  territories  as  units. 
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For  sales  measurements  an  experimental  unit  must  contain  as  part  of  its 
specification  some  time  period  of  observation.   Thus,  a  market  might  receive 
independently  assigned  treatments  in  successive  4-week  periods.   The  unit  is 
then  a  "market  for  4  weeks". 

3. 3  Randomization 

The  random  assignment  of  treatments  to  experimental  units  helps  make 
the  assumptions  used  in  most  statistical  analyses  come  true.   Detailed 
discussions  can  be  found  in  (_3j  ,  l4j  ,  and  [5 J  . 

3 . 4  What  is  measurable? 

To  illustrate  the  type  of  marketing  variable  whose  effect  on  sales 
might  be  measured,  we  list  some  variables  which  have  been  used  in  past 
experiments: 

-  total  advertising  dollars 

-  advertising  dollars  in  specific  media:   for  example,  television, 

newspapers,  and  radio 

-  salesmen's  call  rate 

-  product  appearance 

-  campaign  themes 

-  cooperative  advertising 

-  point  of  purchase  display 

-  store  demonstrations 

-  dealer  contests 

The  basic  question  in  measurability  is  whether  the  experimental  treatments 
will  create  enough  of  an  effect  to  be  detected  amid  the  statistical  variations. 
More  important  to  profitability,  however,  is  whether  the  accuracy  is 
sufficient  to  detect  differences  that  are  both  economically  significant 
and  likely  to  occur.   This  question  can  be  posed  as  a  formal  problem  in 
decision  theory,  but  we  shall  here  approach  it  descriptively.   Our  goal  is 
to  find  standard  errors  of  appropriate  quantities  so  that  the  adequacy  of 
an  experiment  can  be  judged  in  advance. 

Three  categories  of  experiment  will  be  examined:   (1)  spending  experi- 
ments with  fixed  cost  spending  rates,  (2)  spending  experiments  with  variable 


cost  spending  rates,  and  (3)  the  general  two  treatment  comparison. 

1.   Spending  experiments  involving  sales  response  to  fixed  cost  spending. 
These  were  introduced  in  section  2.   We  would  like  to  answer  the 
following  questions  about  the  sales  response  curve: 

(a)  What  is  the  slope  near  the  company's  present  operating  position? 

(b)  What  is  the  curvative,  i.e.,  where  does  diminishing  returns  set  in? 

(c)  How  would  competitors  react  to  changes  in  company  spending  and 
what  is  the  effect  of  their  spending  on  the  company's  response 
curve? 

The  slope  tells  whether  an  increase  or  a  decrease  in  spending  would  be 
profitable.   The  curvature  tells  how  far  such  a  move  could  apparently  be 
made  with  profit. 

The  problem  of  competitive  action  is  quite  serious  and  we  do  not  pretend 
to  solve  it.   It  may  be  that  an  apparently  profitable  increase  in  spending 
rate  would  be  turned  into  a  loss  by  a  matching  increase  from  competitors. 
Experiments  of  the  type  being  considered  are  unlikely  to  be  of  value  in  second 
guessing  the  competition.   However,  they  may  sometimes  reveal  the  effect  of 
one  company's  spending  on  another  company's  sales. 

The  accuracy  of  estimating  slope  has  been  discussed;   we  found  that  in 
a  two  level  experiment 


—  =  estimate  of  slope 


SE(b) 


^  (2.6) 


Curvature  will  be  viewed  as  a  change  in  slope.    Suppose  three  spending 
levels  are  tested: 


spending 
(dol/hh. y 

rate 
r.) 

no.  of 
expt.  units 

ave 

sales  observed 
(dol/hh. yr.) 

low 

'^l 

"l 

^1 

medium 

^2 

"2 

^2 

high  X-  n 

For  simplicity,  we  shall  specialize  to  the  symmetric  case: 


1  =  n   and   x 


Thus  a  medium  rate  with  n„  experimental  units  at  x  ,  is  compared  to  a  high 

rate,   x-  +^,  with  n  units  and  a  low  rate,  x  -A,    with  n  units.   The  same 

statistical  model  is  assumed  as  in  section  2j  sales  in  a  market  are 

given  by: 

s  =  s(x)  +6      dol/hh.yr. 

Let  b. .  be  the  slope  between  the  levels  x.  and  x.  .   Then 
ij  1      J 


12 


23 


"2    1  3    2 

The  interesting  question  is  whether  the  slope  is  decreasing  as  x 
increases. 
Let 


b   -  b   =  change  in  slope 


^ 


estimate  of  c 


23  -  12 
Diminishing  returns  imply  a  negative  c. 

We  want  the  standard  error  of  c*.   Substituting 


V(c)  ,^, 


il 


V(s^)  +  UV(s^)    +  v(' 


^^0 


SE(c) 


^2  [_n  -^  n^   nj 

^b      "2J 


(3.1) 


(3.2) 


As  an  example^  suppose 

"i  =  "- 

10 

^  = 

.01 

n,  =  20 

^^2^ 

.02 

„3=10 

^3  = 

.03 

cr  = 

.02 

(dol/hh.yr 

.) 

(_ 

1 

^^(^2^  =f§f  I^-^ 

1 
20 

2 

=  . 

78 

=  SE(b23 

1 

) 

.01  (dol/hh.yr.) 


A       A 

SE(c)  =  SE(b 


3  "  ir    ~  .01   10  ^  20    =  1 
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Comments:   (1)   As  can  be  seen  in  the  example  or  by  comparing  (2.6) 
with  (3.2),  measuring  changes  in  slope  is  harder  than  measuring  the  slope 
itself.   Published  experiments  that  have  had  some  success  in  measuring 
curvature  have  used  very  large  differences  in  x.   For  example,  Benjamin 
and  Maitland's' — 'best  results  are  for  cases  where  the  ratio  between 
largest  and  smallest  x  values  goes  as  high  as  9/1  and  15/1  . 

(2)   The  split  of  experimental  units  between  n  and  n   in 
the  above  setup  affects  SE(c).   For  a  fixed  number  of  available  units, 

A 

SE(c)  is  minimized  by  setting  n   =  2n  .   This  may  be  verified  by  supposing 

the  total  available  is  N,  substituting  n„  =  N  -  2n  in  (3.2)  and  minimizing 


slope  between  two  chords.   With  three  points  on  the  sales  response  curve, 
a  smooth  curve,  say  a  quadratic,  could  be  fit  through  them  and  such  a  curve 

would  be  a  convenient   representation  of  the  data.   However,  the  standard 

A 

error  of  c  gives  a  measure  of  whether  such  a  fit  is  likely  to  be  meaningful. 

2.   Spending  experiments  involving  sales  response  to  variable  cost  spending. 
A  price  deal  or  dealer  contest  in  which  compensation  is  a  percentage 
of  sales  requires  a  slightly  different  analysis,  since  in  such  cases  the 


promotional  treatment  affects  the  incremental  profit  per  unit.   Our 
objective  is  the  same:   to  calculate  in  advance  a  standard  error  that  will 
tell  us  whether  a  proposed  experiment  has  meaningful  accuracy. 
Here^it  will  help  to  change  our  dimensions  somewhat.   Let 
s  =  sales  rate  in  units/hh.yr . 
m  =  incremental  profit  in  dol/unit 
p  =  profit  rate  in  dol/hh.yr. 
Our  statistical  model  is  that  the  sales  in  an  experimental  unit  is 

s  =  s(m)  +e  (3.3) 

where  £    is  normally  distributed  with  mean  zero  and  standard  deviation  a 
units/hh.yr. 

Consider  a  case  where  the  promotional  treatments  compared  have  incremental 
profits  m  and  m   .   These  might  represent  a  price  deal  and  normal  operations. 
Suppose  n,  markets  receive  m  and  produce  average  sales  s  while  n  markets 
receive  m  and  produce  average  sales  s   . 

Let       'p„  -  ^,    =   estimated  profit  difference 


SE 


(P2  -^i>    =   "^ 


(3.4) 


'2    "1 

The  profit  differences  are  perhaps  more  meaningful  when  expressed  as 
fractions  of  sales  in  dollars.   Let 

s   =  national  sales  rate  (units/hh.yr.) 


normal  national  incremental  profit  (dol/unit) 

1 


then 


SE 


V  m  s   /   s 

V  n  n  /    n 


I— 

I 

?     — 

a 

("A 

1 

K\ 

1 

s 

n 

I  m     ) 

"2 

VJ 

■^1 

(3.5) 


Example;   o/s   =  CV 


.08 


then 


m,  =  .6m 
1      n 

/A    A 

n  n 


2 .  34% 


In  other  words,  using  these  particular  numbers,  if  the  difference  in  profit 
between  the  two  treatments  is  on  the  order  of  5%  of  sales,  we  stand  a  very 
good  chance  of  detecting  the  better  one.   If  the  difference  is  something 
like  17o  of  sales,  we  will  not  obtain  a  reliable  measurement. 


3.   The  general  two  treatment  comparison 

Marketing  alternatives  sometimes  have  little  or  no  cost  difference 
between  them,  as,  for  example,  different  copy  themes  or  different  media 
mixes  within  the  same  budget.   Sometimes  the  treatment  difference  is  a 
single  shift  in  fixed  cost  rate,  as  a  retooling  for  a  product  modification. 
The  omnibus  calculatiow  that  can  be  made  the  basis  for  further  profitability 
comparisons  is  the  computation  of  the  standard  error  of  the  difference  in 
treatment  means. 

Suppose  n   experimental  units  are  given  treatment  1  and  produce  an 
average  sales  rate  s   (say,  in  dol/hh.yr.)  and  n  units  are  given  treatment 
2  and  produce  an  average  sales  rate  s   .   Let  a  be  the  standard  deviation 
for  individual  experimental  units  and  s   the  national  average  sales  rate. 


Then 


estimated  difference  in  sales  between  treatments 
as  a  fraction  of  national  sales  rate. 

1 

2 

(3.6) 


Example:    a/s 


CV  =  .08 
.  =  30 


2.02% 


-  26  - 

Thus  the  treatments  must  produce  sales  differences  which  will  show  up 
against  a  standard  error  that  is  2.027c  of  sales  in  order  to  be 
detectably  different. 

3.5  Formal  Experimental  Designs 

Special  arrangements  of  treatments  on  experimental  units  will  often 
increase  the  efficiency  of  an  experiment.   Two  or  more  variables  of  interest 
can  be  examined  in  the  same  experiment.   Sources  of  variation  that  are  not 
of  interest  but  would  otherwise  obscure  the  treatment  effects  can  sometimes 
be  measured  and  removed.   The  subject  is  a  large  one  and  the  reader  is 
referred  to  the  books  on  the  field^  e.g.  [_3~\    [U~\   and  [_5\    .      We  shall  confine 
ourselves  with  brief  descriptions  of  marketing  applications  for  three  standard 
designs. 

Randomized  Blocks 

Suppose  that  we  wish  to  compare  promotions  A  and  B  by  using  A  in  8 
cities  and  B  in  8  cities.   Instead  of  assigning  the  treatments  at  random 
to  some  list  of  cities^  we  might  break  the  country  up  into  regions^  say^ 
East^  South,  Midwest,  and  Far  West,  and  within  each  region  randomly  assign 
A  to  two  cities  and  B  to  two  cities.   The  presumption  is  that  sales  per 
household  would  be  more  homogeneous  within  a  cuographical  region  than  in 
the  country  as  a  whole.   A  region  is  a  "block".   Because  of  the  balanced 
arrangement  any  additive  sales  effect  attributable  to  the  region  can  be 
estimated  and  removed. 

The  statistical  model  for  the  experiment  is  as  follows;   Sales  for  a 

city  located  in  the  i   block  and  receiving  the  j   treatment  are  given  by: 

s.  .  =  n  +  b.  +  t.  +e 
ij        1     J 

where  s..  is  sales  (say,  in  dol/hh.yr.),  |j,  is  mean  sales,  b.  is  the  differen- 
tial effect  of  the  i   block,  t.  is  the  differential  effect  of  the  j 
treatment,  and  ^  is  a  random  error.   The  Qs   are  presumed  normally  and 
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independently  distributed  with  mean  zero  and  variance  o 

Without  the  block  arrangement,,  random  assignment  of  treatments  would 

2 
have  lumped  the  b.  into  ^  and^  if  b.^  0^  would  have  increased  o     and 

decreased  accuracy. 

Latin  Squares 

A  Latin  square  is  a  particularly  symmetric  design  in  which  several 

treatments  are  simultaneously  compared  and  two  sources  of  uncontrolled 

variation  are  eliminated.   A  good  example  appears  in  the  winter  pear  study 

ri4] 

of  Hind^  Eley,  and  Twining 

Their  Latin  Square  compared  5  experimental  treatments  in  5  cities 
over  5  four-week  time  periods.  The  layout  was  as  follows: 


Cleveland  Baltimore   Milwaukee  Houston  Atlanta 


A  E  D  B  C 

B  C  E  A  D 

E  B  C  5  A 

D  A  B  C  E 

C  5  A  E  B 


1st  4  wks 
2nd  " 
3rd  " 
4th  " 
5th  " 


The  experimental  treatments  were: 

A.  Special  point  of  purchase  displays 

B.  Store  demonstrations 

C.  Dealer  contests 

D.  Media  advertising 

E.  No  sponsored  promotions  by  the  Pear  Bureau 

The  experimental  unit  consisted  of  15  supermarkets  in  each  city  for  4  weeks, 
The  principal  dependent  variable  was  sales  of  winter  pears,  but  sales  data 
was  collected  on  other  fruits.   A  variety  of  extra  independent  variables 
were  recorded,  including  price,  display  space,  and  newspaper  advertising. 
Sales  data  was  collected  by  store  audit. 
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A  Latin  square  can  be  of  any  size  but  the  number  of  columns,  and  the 
number  of  treatments  must  all  be  equal.   Each  treatment  appears  once  and 
only  once  in  each  row  and  similarly  in  each  column.   The  design  removes 
uncontrolled  variation  (to  the  extent  of  an  additive  constant)  in  the  rows 
and  in  the  columns.   In  the  pear  experiment  a  city  effect  and  a  time  period 
effect  were  removed. 

The  statistical  model  assumes  that  sales  for  an  experimental  unit  in 
period  i  and  city  j  under  treatment  k  are  given  by 

where  u  is  the  mean  sales,  r.  a  row  effect,,  c.  a  column  effect,  t   a  treat- 
ment  effect,  and  C  the  usual  random  error. 

In  the  experiment  cited,  there  were  strong  city  effects.   Their  removal 
decreased  o   and  made  the  experiment  more  sensitive. 

A  Latin  square  does  not  estimate  "interactions".   Thus,  suppose  store 
demonstrations  are  particulary  good  in  some  cities  and  not  in  others.   The 
experiment  contains  no  means  of  detecting  any  difference, but  rather  measures 
an  average  effect  over  all  cities. 

We  note  that  a  modification  of  the  Latin  square,  known  as  a  cross-over 
design,  permits  estimates  of  possible  carry  over  effects  in  which  a  treatment 
in  one  time  period  affects  response  in  the  next.   See  flj  and  Qf]  . 

Factorial  Designs 
Factorial  designs  do  estimate  interactions.   Suppose  we  wish  to  compare 
two  television  treatments,  high  and  low,  and  also  two  newspaper  treatments, 
high  and  low.   In  such  a  case  we  would  probably  not  trust  a  model  without 
interactions.   In  other  words,  if  the  difference  between  high  and  low  TV 
treatments  with  newspapers  at  their  low  level  were  57o  and  the  difference 
between  high  and  low  newspapers  with  TV  at  its  low  level  were  4%,  we  might 
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doubt  that  high  TV  combined  with  high  newspapers  would  be  9%.   We  might 
expect  some  diminishing  returns.   A  deviation  from  the  sum  of  the  single 
effects  would  be  called  an  interaction. 

The  factorial  arrangement  for  this  experiment  may  be  depicted  as  follows: 


TV 


High 

Low 

High 

Low 

News, 


Each  cell  would^  hopefully^ contain  several  experimental  units.   In  this  type 
of  experiment  the  unit  would  probably  be  a  metropolitan  market.   The  "factors" 
here  are  television  and  newspapers.   Each  factor  has  two  or  more  "levels"^ 
in  this  case^  "high"  and  "low". 

The  characteristic  of  a  complete  factorial  experiment  is  that  each 
level  of  a  factor  appears  in  combination  with  every  level  of  every  other 
factor. 

The  statistical  model  for  the  above  experiment  assumes  that  sales  in  a 
unit  receiving  TV  at  level  i  and  newspapers  at  level  j  is: 
3.  .  =  ,  +  a.  +  b.  f  c.  .  +C- 

where  |j.  is  mean  sales,  a.  the  main  TV  effect,  b.  the  main  newspaper  effect, 
C.  the  TV-newspaper  interaction,  and  £  the  random  error  term. 

4.0  Analysis 
An  experiment  is  analyzed  according  to  the  statistical  model  presumed 
to  underly  the  data.   An  formal  design  has  a  corresponding  model.   Adding 
extra  explanatory  variables  to  the  model  is  frequently  desirable,  as  has  been 
remarked. 
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A  good  variation  on  previous  period  sales  as  an  explanatory  variable  can 
be  constructed  if  a  considerable  time  series  of  post  sales  is  available  for 
each  experimental  unit.   A  time  series  analysis  of  post  sales  can  be  made  for 
each  unit  separately  and  an  extrapolation  of  sales  into  the  experimental 
period  made.   The  extrapolated  sales  can  be  used  as  an  added  explanatory 
variable  in  place  of  prior  period  sales. 

Sometimes  information  is  available  to  show  that  certain  experimental 
units  come  from  populations  with  higher  variance  than  others   For 
instance,  small  cities  usually  have  somewhat  higher  coefficients  of  variation 
than  large  cities.   In  such  cases,  the  efficiency  of  estimation  can  be 
increased  by  weighting  each  observation  inversely  proportional  to  its  variance. 
4. 1  Analysis  of  Experimental  Designs  by  Regression 

Books  on  experimental  design  give  different  computational  formulas  for 
each  new  design.   These  are  of  interest  for  a  variety  of  reasons,  and  when 
calculations  are  to  be  done  on  a  desk  calculator  these  formulas  are  almost 
essential.   However,  whenever  a  standard  experimental  design  is  viewed  as  a 
fixed  constants  model  it  becomes  a  special  case  of  the  general  linear  hypothesis 
and  as  such  can  be  analyzed  on  a  computer  by  a  multiple  regression  program. 

The  analysis  of  an  experiment  by  a  multiple  regression  program  will 
give  an  estimate  of  each  constant  in  the  design  and  a  standard  error  for 
it.   In  addition  most  programs  will  give  the  covariance  between  each  pair 
of  estimated  constants  or  else  information  from  which  it  can  be  easily 
deduced.   However,  a  regression  program  does  not  ordinarily  produce  directly 
the  information  required  for  a  table  of  analysis  of  variance. 

Analysis  by  regression  makes  it  possible  to  use  one  computer  program 
for  all  different  classes  of  designs.   Regression  will  automatically  accept 
data  with  missing  observations  or  extra  observations  and  treat  designs  not 
found  in  standard  texts.   When,  as  is  usual  in  marketing,  there  are  important 
explanatory  variables  outside  the  design  proper  (e.g.  last  year's  sales), 
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their  inclusion  in  the  analysis  is  very  simple. 

To  illustrate  the  use  of  regression^  we  analyze  an  experiment  consisting 
of  a  pair  of  3  x  3  Latin  squares.   The  data  are  entirely  made  up.  In  fact, 
the  residual  error  has  been  made  absurdly  small  in  order  to  show  that  the 
analysis  does  indeed  recover  the  constants  used  in  making  up  the  data. 
A  pair  of  3  x  3  Latin  squares 

12  3  4  5  6 


I 

II 

III 


A B C 

B C A 

CAB 


I 

II 

III 


A  B C 

C A B 

B  C  A 


Suppose  that  there  are  three  experimental  treatments  to  be  evaluated, 
A,  B,  C;  six  markets  in  which  the  test  is  being  conducted,  1,  2,    3,  k,    5,  6; 
and  three  successive  time  periods  (perhaps  months)  I,  II,  III.   In  each 
market  in  each  time  period  some  one  of  the  three  treatments  is  administered 
as  indicated  by  the  cells  in  the  above  squares.   The  dependent  variable 
is  sales. 

The  observations  are  presumed  to  have  been  generated  by  the  market 
(or,  in  our  example,  by  us)  according  to  the  model 


where 


.      „  +  a.  +  p.  +  d  +  g 
ijk        1    J    k    *- 

sales  in  the  i   market  area  in  the  j   time  period 


(4.1) 


ijk 


under  treatment  k. 
m  =  mean  sales  for  all  markets 

,.  =  differential  effect  on  sales  of  i'' "  market  area 

,th 


th 


time  period 


^   =  random  error 
All  units  would  be  the  same,  say,  thousands  of  dollars.   The  random  error 
is  assumed  to  be  independent  from  cell  to  cell  and  to  be  normally 
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2 
distributed  with  zero  mean  and  some  variance  a 

An  experiment  is  usually  oriented  toward  measuring  differences, 
e.g.  the  difference  between  treatments  A  and  B  on  sales.   This  would  be 
d„  "  d .  .   A  constant  added  to  each  d  would  not  affect  the  difference. 
By  convention,  it  is  presumed  that  a  constant  has  been  added  to  each  d 
so  that  the  sum  of  the  d,  is  zero.   The  same  remark  applies  to  the  a. 
and  the  p  .  • 

We  synthesize  some  hypothetical  data. 


1 

= 

-.3 

2 

= 

-.2 

3 

= 

-.1 

4 

= 

5 

= 

6 

= 

'II 


III 


-1.80 

.12 

1.92 


m  =  3.57 

From  the  above  table  we  can  construct  the  expected  value  of  sales  for 
any  cell  in  the  squares.   For  example,  the  expected  value  for  1,  1,  A  is 

^^^1,1, A)  =-+Pi+a^+dA 

=  3.57  +  .1  -.3  -1.80  =  1.57 
Continuing  in  a  like  manner^  we  develop  the  mean  sales  in  each  cell  as 
follows: 


I 

II 

III 


1.57A 

3.35B 

5.49*^ 

3.15S 

5.29"^ 

1.67^ 

5.09^ 

1.47^ 

3.25^ 

1.97* 

3.75^ 

5.89^ 

5.59^ 

,1.97^ 

3.75^ 

3.45« 

5.59^ 

1.97^ 

To  complete  the  synthesis,  a  random  normal  number  of  mean  zero  is  added  to 
each  cell.   A  very  small  standard  deviation  (a  =  .01)  has  been  used  to 
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make  it  clear  that  the  analysis  does  recover  the  original  numbers.   The 
final  "observed"  sales  data  are: 


1.585^ 

3.355^ 

5.494^ 

3.142^ 

5.286^ 

1.676^ 

5.089*' 

1.471^^ 

3.249^ 

1.989* 

3.765^ 

5.899^ 

5.583^ 

1.975^^ 

3.740^ 

3.450^ 

5.601^ 

1.966^ 

I 

II 
III 


We  proceed  to  analyze  the  experiment  by  multiple  regression  in  the 
following  steps.   First  we  set  up  a  model  with  dependent  variable  y  and 
independent  variables  u.,  v.  ,    and  w  .   Then  we  change  independent  variables 
slightly  and  rename  them  x..  ^  x„,    ...  to  conform  to  usual  regression 
program  notation.   A  table  of  input  data  for  such  a  program  is  presented. 
The  output  is  given,  showing  means  and  standard  errors  for  the  constants 
of  the  model.   Finally  we  show  how  to  compute  the  standard  error  of  the 
difference  between  two  of  the  estimated  constants. 

Consider  first  the  regression  model. 


(4.2) 


+  Va  -^  ^b"b  ■"  ^c"c 


where  each  data  point  (cell  observation)  is  to  have  associated  with  it  a 

set  of  values: 

y  =  sales  for  the  cell  observation 
:o  market  1 


il  if  cell  refers  t< 
1   (^0  otherwise 


-  y,  - 

1  if  cell  refers  to  market  6 


'1  if  cell  refers  to  time  period  I 
I   10  othervjise 


(1  if  cell  refers  to  time  period  III 
III   [O  otherwise 

il  if  cell  refers  to  treatment  A 
'a  ~  \0  otherwise 


1  if  cell  refers  to  treatment  B 
0  otherwise 

e  =  random  error 

A  regression  on  these  variables  will  fail.  This  is  because  certain 
variables  can  be  expressed  as  linear  combinations  of  other  variables  and 
as  a  result  the  matrix  to  be  inverted  in  solving  for  the  coefficients  is 
singular.   We  can,  however,  get  rid  of  3  variables  by  recalling  that 

Za.  =  0        Zp.  =  0        ^\  "  ° 


^11  =  -  ^Pl  ^PlI)  ^^-3^ 


^c  -  (^A  +  '^b) 


Substituting 


a^(u^  -  V  ^  •••  ^^  ^"5  -  "6^ 


d(v-v    )+d('v    -v    ■) 

^l'  I       III'      '^II^  II       III'' 


+  d^(w^  -  w^)  +  d^(w^  -  w^) 


Now,  letting 
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^3  -  > 
We  obtain  the  regression  model 


II    III 


y  =  m  +  a  X 


1  1 


VS  -^  Pl"6  -^  PlI^  +  Vs  -^  V9  -^  "   ^"^-"^^ 


where  for  each  observation 


y  =  sales 


if  cell  refers  to  market  1 
6 
otherwise 

if  cell  refers  to  market  5 
6 
otherwise 

if  cell  refers  to  period  I 

"  III 

otherv/ise 

if  cell  refers  to  period  II 

"  III 

otherwise 

if  cell  refers  to  treatment  A 
C 
0  otherv7ise 

if  cell  refers  to  treatment  B 
C 
0  otherv/ise 


The  regression  will  not  estimate  a     ,    ?        ,    d     directly  but  they  are 

6    III   C 

given  by  substitution  into  (4,3) • 
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In  this  experiment  there  are  18  observations.   The  regression  input  can 
be  tabulated  as  follows  (blanks  are  zeros) : 


Obs. 

No. 

y 

::i 

^2 

!3 

!it    !5 

!i    !2 

!8 

^ 

1 

1.585 

1 

1 

1 

2 

3.142 

1 

1 

3 

5.089 

1 

-i    -1 

-1 

-1 

4 

3.355 

1 

1 

5 

5.286 

1 

-1 

-1 

6 

1.471 

1 

-1     -1 

1 

7 

5.494 

1 

-1 

-1 

8 

1.676 

1 

1 

9 

3.249 

1 

-1     -1 

1 

10 

1.989 

1 

1 

11 

5.583 

1 

-1 

-1 

12 

3.450_ 

1 

-1     -1 

1 

13 

3.765 

1 

14 

1.975 

1 

15 

5.601 

-1     -1 

-1 

-1 

16 

5.899 

-1 

-1 

-1 

-1    -1 

-1 

-1 

17 

3.740 

-1 

-1 

-1 

-1    -1 

1 

18 

1.966 

-1 

-1 

-1 

-1    -1 

-1     -1 

1 

The  output  from  a  typical  regression  program  is  shown  below. 
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Analysis  of  Experimental   Design  by  Regression 


C0VARl\NCe    MATRIX 


1     1.666666E-0l-3»333333E-(H 

1    1.666666E-01  -W666666E-GV   3»333333E-04- 

rfr66666E-0l     l»666fi66C    01    Ii666666E-01    3.33^^33E-01 


1  •  666666E-01     1^  666666E-<B^  •  666ft66E-04-4  •666666E-01    3*3  3333  3E  -0 1 

O.OOOftOOE-oq    f)j0OOOf»OE-99^-0TO00O00E— 99-  0»00000'^E-99    0«00r»00nF-99 


-7 ^-rr^rrtm)O0OC"99    OtOOOnOOC-99    0i000(K)0C-99    0»000Oe^E-')9    O»f>f>A0OeF' 


R 
9 

6 

7 
-8- 

9 
10 


1  n«Or>O00nE'-99  0»OOO0a0E-99-fy»aOOf)00E— 99-0^rOO000OE-99  ft.OOnooOE-99 
1  O.OftOOOOE-99— OTOOOOaoE-99~<y»©OOOOOt-99-O»0OOOOOE-99  O«OOO000F-99 
-1-9y966e8flC    02    Oig9»<»^<»C    02-6.56e860E-00-3»23e886E-02-l>»66666E-02 

6    6t666666E-0l-  

6    3.333333E-01  -6»6«6666E-el~  

6    0 • OOOOOOE -99  ^i OOOOOOE-99  -3» ^33^33E-0l    6^666666E-(M 

6    7*005555E-02-^20OOO0E-O£-1^238533e-OO-<»-8061llE-Ol   2,36?135E-00 


MEANS  

1     ,nnof)n0F^99^-rrKJO00O€-99    irK)Ot)O0f-99-ia000O0E-99    •00r>000F-99 
a    ,000000F-99    «000000E*99    iOOOOOOE-99    <000000C-99    t3573^f-HH- 

STANDARO   DEVlATI0r4S — — 

1— r9 7 7300C-00    ■!)77990C-00    »577350E    00    >57735oE    00    »S77350E-00 

6    i816»96E--Oa"r^^^»9frE-ee-»816<>96E~0e^846»96E-ee   >l&3692E-f01 

— ^0RReLATlgN    MATRIX 


r500000 U  000000 

»9e68e3 -.54^2365 


1*000000 
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MULT.    R    »      »999^T- 


-StjM   0F    RE5I0UALS    ■ 
TABLE   0F    RESUL 
VAR  B-C0EF 


ST»    ERRT-gF-^5Tr-» .0044» F|- 9* — B^ 

OtOOOO* GUM   gr    SQUARED   RESIDUALS    " — 


.0001 


-&TrERR.&— 


RTIAL 


BET/ 


— fNVERSC    MA^R-H( 

0      0    5»555555E-02 — 

—  10   0*000000E-99  -2W77777E-04 

2 0-&TO0OO00C-99-5.555555C  02  g.777777E-01 


3  0  O*000000e-«»9-5.555555^E-<y2-5r555555E-02  2«^77777E-al 

4  -  0   0*00000nF-99-5r555555E-O2-5i555555E^2-5,'>555'»5E-O2-2»777777E-01 
-5 — 0-&T<K)OtM)OE-99-Dt;>!J5553C-02    bwbbbbbbZ    02    5.555555E    0J-5»5&55^5^E-O2- 

h      O  -n.0ft0O0nF-O9-A«t)f>O0t>0E-99-^yTOOOOOOE^99^-0T0O0000E-99-fyrO0000PE-99 
7      0    0•000000F-99-aTtroOOO0E-99~<^*O0OO00E-99-0.0OOO00E-99-0T00O0O0E-O9 

-fl n    n»0n000MF-99    O»000n00E-99    0*000000E-99    OtOOOOOOC-99    0»00000OF-99- 

9       rm«0O0O0'>E"-99    0«  OOOOOOE-99— TTj 

5  5^   2*777777E-Ol 

-t 9    O.OOOOOOE-99    lillllllC-Ol 

7  5   0.0000 00E-9^-5T5555^55E-O2-4^r««ii€--ei-  

8  "J    0^. 0O0000F-99-0*00000O€-99-OTOOm>OOE-99-|»H  1 1 HE-0-1 

-9 ^    O.OOOOOOE-99    0>000000[:-99    0»0O0O00C-99-5»555555E-02    l«llllllE-0^ 


'99^-<tTnfYf»f)OftE-99-n*^>f>'>O0rtF-99 


Although  standard  errors  are  tabulated  by  the  program  for  the  constants 
of  the  regression,  an  additional  step  is  required  to  find  the  standard  error 
of  a  difference  between  two  coefficients.   As  an  example  we  find 

sE(d;  -  \)  W  v(^^  -  d^) 

where  SE  stands  for  standard  error,  V  for  variance  and  the  carat  denotes 

the  regression  estimate.   Using  the  general  formula  for  the  variance  of  a  sum 

V(I   -  d)        =   V(2  )  +  V(d^)  -  2  CovC'd  ,d  ) 
A    B  A       B  Ad 

Simply  squaring  the  tabulated  standard  error,  we  find  that 

V(d.)  =  V(d„)  =  1.96  X  10"^ 
A       B 

The  covariance  term  can  be  found  from  an  appropriate  inverse  matrix. 

The  particular  inverse  printed  out  varies  somewhat  from  program  to  program. 

Here  the  inverse  of  the  sum  of  cross  products  matrix  is  given.   Consider 

two  arbitrary  independent  variables  x.  and  x.  with  coefficients  b^  and  b. 

Then 

A  A  2 

Cov(b.,b.)  =  (i,j  element  of  inverse) (s  ) 

where  s   =  the  standard  error  of  estimate  of  the  regression.   Thus, 

Cov(d.,d„)  =  -(.0555) (.0044)^  =  1.08  x  lO'^ 
A   B 

and  so    SE(d^  -  d^)  =  .0025 

If  the  inverse  of  the  correlation  matrix  is  printed  out,  the  covariance 

calculation  is  slightly  different.   Let  r   =  the  (i,j)  element  in  the  inverse, 

a     be  the  standard  deviation  of  x  ,  and  n  the  number  of  observations.   Then 
i  1 

Cov(b,,b.)  =   r    2 

11  o 

no.a. 
One  task  remains.   We  can  find  a  ,  P,..,^  and  d   from  (4.3)  but  their 
standard  errors  and  possibly  their  covariances  are  also  desired.   The  problem 
is  trivial  here,  since  all  the  standard  errors  for  the  a's  are  the  same, 
all  the  pVs  the  same,  etc.   In  the  general  case,  the  regression  output  can 
be  manipulated  to  obtain  the  numbers^  but  an  easier  way  is  to  run  two  more 
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regressions.   A  regression  with,  say,  a  ,  P-rr-"  ^""^  ^t,    ^^    '-^^  constants 

J    II        B 

A     y^  '^ 

eliminated  will  produce  a,,  p    ,  and  d   and  their  standard  errors.   The 
data  to  calculate  all  covariances  except  cov(a  ,a  ),  Cov(p   ^P   );  and 

A   A 

Cov(d  ,d  )  is  also  easily  available.   Data  for  the  latter  can  be  produced 
B   c 

by  a  regression  having  a,,    p   ,    and  d  eliminated. 

Another  variation  of  the  regression  can  be  useful.   Suppose  we  are 
specifically  interested  in  the  differences 


Then,  instead  of  eliminating  a,  in  (4.2)  by  converting  the  u's  to  the  x's, 

we  can  resolve  the  singularity  caused  by  the  a's  in  a  more  useful  way.   Let 

X   be  a  variable  that  takes  on  the  value  unity  for  every  observation.   Then 
o 

Substitution  into  (4.2)  produces  a  regression  model  with  a  constant  term  of 

m+  a,x   and  the  desired  differences  as  coefficients  of  u  variables. 
6  o 

In  setting  up  the  variables,  x   is  not  introducedj  the  constant  term  will 
automatically  include  a,  .   The  v  and  w  singularities  can  be  resolved 
similarly  or  by  the  x  method.   When  the  regression  is  run,  the  standard 
errors  of  the  differences  we  have  set  up  can  be  read  off  directly  as  standard 
errors  of  the  corresponding  coefficients. 

In  summary,  multiple  regression  computer  programs  provide  a  general 
means  of  analyzing  experiments  to  estimate  parameters  and  ilicir  standard 
errors.   To  use  multiple  regression  in  this  way  it  is  necessary  to  knov;; 
(1)  the  basic  statistical  model  being  assumed)  (2)  how  to  set  the  model  up 
in  regression  form;  (3)  how  to  determine  covariances  of  the  regression 


coefficients  from  the  computer  output;  (4)  how  to  express  variances  of  sums 
of  random  variables  in  terms  of  their  variances  and  covariances. 

An  excellent  use  of  this  type  of  analysis  is  in  planning  an  experiment. 
Frequently  designs  are  proposed  that  differ  from  standard  form.   Perhaps 
some  combination  of  treatments  is  not  operationally  possible.   What 
would  happen  if  that  combination  were  omitted?   A  dry  run  of  the  analysis 
will  be  helpful  in  deciding  whether  important  accuracy  is  lost.   Such  a 
dry  run  could  be  conct,.!.:ci:t;(:  r.s  r.bcve^  ci,,  betLe/,  live  historical  data 
might  be  used^  perhaps  adding  in  effects  of  the  approximate  size  anticipated. 


5.0  Accuracies  Obtained  in  Practice 

Quite  a  variety  of  sales  experiments  have  been  conducted  ,over  the  past 
10  years.   As  might  be  expected^  most  of  them  are  unpublished,  although  the 
Department  of  Agriculture  Series  is  a  happy  exception.   The  writer  has  collected 
together  a  certain  amount  of  design  and  accuracy  information  from  a  number  of 
experiments.   The  data  is  displayed  in  Table  5.1 

The  purpose  of  the  table  is  to  give  an  indication  of  what  has  been  done 
in  the  past  and  particularly  to  show  the  range  of  standard  errors  that  have 
been  encountered.   The  coefficient  of  variation  for  an  experimental  unit  is 
a  fundamental  design  parameter  for  picking  sample  size.   The  standard  error 
of  the  difference  in  treatment  means  shows  how  big  a  sales  effect  the  treat- 
ments must  produce  in  order  to  be  detected. 

The  types  of  experimental  design  that  have  been  used  include:   randomized 
blocks,  Latin  squares  with  and  without  change-over,  and  factorial  designs. 

The  experimental  treatments  have  included  total  advertising  expenditures, 
advertising  expenditures  in  specific  media  (especially  television  and 
nespapers) ,  product  appearance,  point  of  purchase  display,  and  promotional 
themes. 

Types  of  sales  measurement  have  included  store  audits,  factory  shipments, 
and  consumer  surveys. 

Direct  mail  experiments  have  been  excluded  from  the  table. 


Table  5.1 
^{•'DesigP  Qata  from  Past  Experiment: 
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check  for  residual  effects. 

(3) 

CV  =  coefficient  of  variation  =  estimated  standard  ( 

deviation 

for  an 

experiment  uiiit/average  sales  per  experimental  unit.   Where  additional 
explanatory  variables  were  used  in  regression  or  covariance  models^  the 
adjusted  standard  deviation  (standard  error  of  residuals)  is  used. 
(4)   SE  =  standard  error.   Where  several  different  treatments  were  tested^ 

the  standard  error  of  the  difference  between  some  important  pair  of  means 
is  given.   For  continuous  variables  spread  over  a  range^  an  attempt  has 
been  made  to  compute  the  standard  error  for  a  comparable  two  level 
experiment. 


6.0  Adaptive  Allocation 

We  return  to  the  question  raised  by  Figure  1  (page  4):   How  can  we 
design  more  effective  systems  for  controlling  marketing  variables?  The 
elements  of  a  control  system  should  include  a  model  of  sales  response^  a  means 
of  allocating  market  effort  on  the  basis  of  the  model^  and  devices  for  keeping 
the  model  up  to  date.   Such  devices  may  well  include  the  deliberate  per- 
turbation of  marketing  variables  to  facilitate  response  measurements. 
In  addition^  data  would  be  collected  on  such  quantities  as  product  avail- 
ability^ consumer  attitudes,  and  competititive  activities. 

Every  company  obviously  has  some  procedure  for  controlling  marketing 
variables,  but  usually  the  relationship  between  the  data  input  and  the  setting 
of  the  marketing  variables  is  not  at  all  formally  specified.   Our  interest  is 
in  specifying  appropriate  relationships  and  appropriate  data  inputs  and  in 
studying  their  effect  on  overall  company  performance.   Our  presumption  is  that 
by  careful  systems  design  we  may  be  able  to  achieve  better  performance  than 
is  achieved  now. 

Formal  systems  design  in  this  sense  appears  to  be  a  good  way  off.   What 
we  shall  do  here  is  set  up  and  solve  a  simple  model  embodying  some  of  the 
ideas  that  seem  relevant  to  the  larger  problem. 

The  model  may  be  described  briefly  as  follows:   company  sales  (and  there- 
fore profits)  are  a  function  of  a  marketing  variable  x,  say,  a  rate  of  spending 
money  of  promotion.   The  sales  response  to  x  changes  with  time.   The  change 
takes  the  form  of  a  changing  parameter  in  the  sales  response  function.   The 
value  of  the  parameter  is  estimated  in  each  time  period  by  an  experiment. 
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The  promotion  rate  for  each  period  is  picked  by  using  information  about  the 
parameter  as  collected  in  past  experiments.   Our  problem  is  to  determine  how 
to  design  the  experiments  and  how  to  use  the  resulting  information  in  picking 
the  promotion  rate. 

6.1   Profit  Model 

Let       s  =  sales  rate  (dol/hh.yr.) 

X  =  promotion  rate    " 

p  =  profit  rate       " 

c  =  fixed  cost  rate   " 

m  =  gross  margin^  the  incremental  profit  as  a  fraction  of  sales 
Then 

p  =  ms  -  X  -  c  (6.1) 

We  shall  suppose  that  at  some  fixed  point  in  time  there  is  a  sales  response 
curve  of  the  general  shape: 

s 
sales  rate 


promotion  rate 
Presumably^  the  curve  can  be  approximated^  at  least  near  the  current  operating 

pointy  by  a  quadratic  function  of  x: 

2 
s  =  a  +  px  -  rx  (6.2) 

The  parameters  a,  P^  and  f  are  constants  at  a  fixed  point  in  time  but  some  of 

them  may  vary  with  time. 

The  value  of  x,  say  x*;  that  maximizes  profit  is  easily  determined; 

mrp-2yxj-l   =  0 

P    -    1/m 


d£ 

dx 


2y 


(6.3) 


i(x*)  =  maximum  profit  rate  (dol/hh.yr) 
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p(x)  =  profit  rate  using  x  (dol/hh.yr.) 

,jf(x)  =  p(x*)  -  p(x)  =  loss  of  profit  rate  if  x  is  used  instead  of  x^'  . 

/(x)  =  m  |p(x*  -  x)  -^(x*   -X  )   -  (x*  -  x)  which  after  manipulation 
and  use  of  (6.3)  becomes 

/(x)  =  mJ'Cx  -  x*)^  (6.4) 

6. 2  Model  for  Changes  in  Sales  Response 

If  a^  P,  and  ^  were  known,  we  would  set  x  =  x*  and  obtain/^(x*)  =  0  . 
However,  the  parameters  are  presumably  fairly  difficult  to  measure  and  we 
ordinarily  expect  to  come  up  with  some  non-optimal  x  and  therefore  to  incur  a 
relative  loss. 

If  the  parameters  did  not  change  with  time,  we  would  want  to  put  a  big 
effort  into  measuring  them  right  away^  because  the  extra  profit  from  increased 
accuracy  would  extend  far  into  the  future.   However,  it  is  difficult  to  believe 
that  in  practice  the  parameters  stay  constant.   Thus,  for  example,  competitive 
activity,  product  changes,  and  shifts  in  economic  conditions  lead  us  to  expect 
shifts  in  response.   Consequently,  an  expensive  effort  to  learn  the  parameters 
immediately  cannot  be  justified.   On  the  other  hand,  the  parameters  may  change 
fairly  slowly  with  time,  in  which  case  some  effort  is  worthwhile.   In  each 
time  period  new  information  is  collected,  combined  with  the  old  and  used  to 
set  operations  in  the  immediate  future. 

To  set  up  a  fairly  simple  model  of  changing  sales  response,  we  shall 
suppose  that  a  and  p  change  with  time  but  (^  does  not.   At  a  fixed  period 

in  time  we  assume  that  national  sales  rate  for  the  product  is: 

2 
s  =  a  +  px  -J-x  (6.5) 


and  that 


a  is  a  random  variable  with  high  variance  from  one  time  period  to  the 
next  J 

P  is  a  random  variable  that  changes  with  time  as  specified  below, 
a  and  p  are  independent 
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y  is  a  known  constant 

^s  implied  by  (6.3)  and  as  will  be  verified,  the  information  available 
about  a  does  not  directly  affect  the  optimal  x.   However,  information  about  a 
may  make  it  possible  to  learn  more  about  p  in  an  experiment.   The  assumption 
of  high  variance  tor  a,    (which  may  well  be  fairly  realistic),  simplifies  the 
statistical  analysis  by  removing  a  as  a  contributor  to  information  about  p.) 

The  parameter  p  will  be  assumed  to  be  generated  by  a  random  walk.   One 
possibility  is 

P(t)  =  p(t-l)  +€(t) 

2 
where     ^(t)  =  a  random  variable  with  mean  =  0  and  variance  =  a 

P 

We  shall  assume  £(t)  is  normally  distributed  and  independent  of  previous  values 
of  p  and£ 

The  difficulty  with  the  above  random  walk  is  that  p  is  likely  to  wander 
unrealistically  far  from  its  starting  value.  Therefore  we  shall  hypothesize 
a  long  run  average  value  and  a  tendency  for  p  to  return  to  that  value. 

Specifically,  let 

o 
P  =  the  long  run  average  value  of  p(t) 

k  =  a  constant,  0  <  k  <  1  . 
We  assume 

ti(t)  =  kp(t  -  1)  +  (1  -  k)  p°  +eu)  (6.6) 

Thus,  a  plot  of  p(t)  vs.  t  might  look  as  follows: 


i(t) 
„o 


:^. 


^ 


n_ru, 
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Several  properties  of  the  p  process  are  noted: 

E(p(t))  =  k  E  (p(t  -  1))  +  (1  -  k)  p°  (6.7) 

This  is  a  difference  equation  in  the  expected  value  of  p(t) .   The  steady 
state  solution  is 

E(p(t)j  =  p°  (6.8) 

as  implied  by  our  verbal  definition  of  p 

v(p(t))  =  k^v('p(t  -  1)^   +  a^  (6.9) 

The  steady  state  variance  is 

v(p(t))  =  -^  (6.10) 

6. 3  An  experiment  on  p(t) 

Although  national  sales  rate  is  given  by  (6.5)^  individual  markets  are 
assumed  to  differ  from  one  to  another  at  a  fixed  t  according  to 

s  =  s(x)  +  €  (6.11) 

where     s  =  sales  rate  in  the  market  (dol/hh.yr.) 
s(x)  =  sales  rate  from  (6.5)         " 
^  =  a  random  variable  for  the  market  " 

We  taket  as  normally  distributed^  independent  from  market  to  market  and  as 

2 
having  mean  =  0  and  variance  =  a 

An  experiment  to  estinate  p(t)  is  presented  in  the  following  sketch: 
/N  n  markets      ■'■ 

sf  '  ' 


1     o     2    x(t) 

At  a  given  t^  suppose  that  we  have  picked  a  promotion  rate,  x  (t),  that 

is  in  some  sense  best.   This  is  used  everwhere  except  that  in  n  markets  a 

deliberately  low  value,  x  ,    is  used,  and, in  another  n,  a  high  value,  x  ,  is 


49    - 


We  choose 

X,    =  X   (t)    -  ^/2 

'  °  (6.11) 

X-   =  X   (t)  +A/2   . 
2  o 

Let   s,    and  s_  be   the  observed  mean  sales   rates   in  the  markets   at   x     and  x 
12  12 

respectively.      An  estimate   of   p(t)    can  be   computed   from  the  experimental   data. 


:t)  A~'2-'^2)-(\W) 


Let  p(t,  .  s  -   ^^'^'     '        '    '  (6-12) 

This  will  be  called  the  "experimental  mean."   Let 

V  =  V(^(t)  ) 

From  (6.11);  (6.5) ^  and  (6.12)  we  see  that^  given  p(t),  p(t)  is  normally  distributed 

with  mean  and  variance 

E(p(t))  =  p(t) 

(6.13) 

2  a^ 

v  = 7 

n  ^^-^ 

Notice  that  v  does  not  depend  on  p(t)  or  t. 

The  experimental  result  (6.12)  does  not  represent  all  our  information 

about  p(t)  .   Even  before  doing  the  experiment,  we  had  the  information  developed 

in  previous  experiments.   The  information  will  be  summarized  in  a  prior 

distribution  of  p(t)  .   This  distribution  will  be  taken  to  be  normal  with 

E'^p(t^  =  prior  mean  of  p(t) 

v'  =  prior  variance  of  p(t) 

At  the  beginning  of  period  t,  when  the  promotional  rate,  x(t),  is  to 
be  set,  we  have  only  the  prior  distribution  of  p(t)  .   At  the  end  of  t,  the 
experimental  results  are  available  and  we  can  construct  a  posterior  distribution. 
The  additional  information  about  p(t),  however,  is  of  no  help  in  t  although 
it  will  be  useful  in  t+1  . 

6,4  Decision  Rule  for  Setting  x(t) 

We  propose  a  simple  decision  rule  for  setting  the  promotion  rate  in  each 
time  period. 
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Choose  a  number^  a,    such  that  0  <  a  <  1  .  At  each  t  compute  E'^p(t^  frc 


E'(p(tj)  =  a  E'(p(t-1))  +  (l-a)'p(t-l) 


then  compute 


X  (t) 


(p(t))  -  1/m 


The  decision  rule  is  to  set 
x(t)  =  X  (t)  . 


(6.14) 


(6.15) 


(6.16) 


Suppose  the  process  starts  at  t  =  1  .   It  is  necessary  to  pick  a  value 
E'(p(0))  but,  thereafter,  x(t)  is  set  mechanically  by  the  rule.  /Since  (6.14) 
is  an  exponential  smoothing  process,  the  effect  of  E'(p(0))  on  E'/p(t5)  and 
thence  x(t)  decreases  exponentially  with  t . j 

Notice  that  the  rule  as  stated  makes  no  assumptions  about  the  under- 
lying process  generating  a(t)  and  p(t)  and,  if  we  think  of ^  as  an  arbitrary 
positive  constant,  the  rule  is  not  tied  down  to  any  specific  sales  response 
process.   The  rule  can  be  applied  to  any  situation  in  which  the  experiment  of 
Section  6.3  is  performed,  provided  that  somebody  is  willing  to  pick  a_ and  ^ 
plus  E'{  P(0))  and  the  experimental  design  parameters  A  and  n. 

The  behavior  of  the  decision  rule  will  be  clearer  if  we  express  it 
somewhat  differently.   Putting  (6.12)  into  (6.14)  and  the  latter  into  (6.15)  gives 


X  (t) 


1/m 


+  (1-a) 


(t-1) 


+  Iil2l 


2^ 


h 


(S2   -    Sj^)    +      (x^      - 

.,s 

1/m 

2^A 
^(X2  +  X^)(X2   -   x^) 

1/m 

J 

zi" 

A 

Then,  using  (6.11)  and  making  the  time  period  explicit  for  s„  and  s   ,  we  obtain 


X  (t)  =  X  (t-1)  +  -^^ 
o       o         2<5' 


Uy  [l 


The  quantity]  S2(t-1)  -  s  (t~l) 


% 


So(t-l) 


(t-1) 


1/m 


(6.17) 


is  an  experimental  estimate  of  the  slope 


of  the  chord  of  the  sales  response  curve  between  x  and  x   . 


If  the  slope 
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is  greater  than  1/m,  the  implication  is  that  x^  is  more  profitable  than  x  :  if 
the  slope  is  less^  the  reverse  is  true. 

Thus  the  decision  rule  says  to  use  in  t  approximately  the  same 
promotion  rate  as  in  t-1  but  adjust  it  upward  or  downward  depending  on  whether 
the  experimental  evidence  from  t-1  suggests  a  slope  greater  or  less  than  1/m  . 
The  amount  of  the  adjustment  is  controlled  by  the  constant  (l-a)/2y. 
If  the  constant  is  large^  the  promotion  rate  will  be  sensitive  to  the  most 
recent  experimental  results;  if  the  constant  is  small,  insensitive. 

Although  an  appropriate  choice  of  constants  is  critical  for  reasonable 
operation,  the  general  form  of  (6.17)  provides  an  adaptive  control  system 
that  might  be  expected  to  work  fairly  well  for  a  variety  of  underlying  sales 
response  mechanisms.   One  might  hope  that  if  the  constants  are  chosen  with  one 
mechanism  in  mind,  they  might  work  satisfactorily  with  other  mechanisms  not 
too  different.   Whether  or  not  this  is  so  in  a  specific  case  is  a  question 
that  can  be  explored   oy  means  of  simulation. 

For  the  sales  response  model  being  assumed  we  now  wish  to  motivate 
the  decision  rule  more  carefully  and  then  go  on  to  pick  values  for  a,    n,    and 
which  are  optimal  or  close  to  optimal. 

6.5  Choosing  a 

Consider  first  the  problem  of  finding  the  posterior  distribution  of 

p(t)  given  the  prior  distribution  the  the  experimental  results.   Let 

E'(p(tj)  =  prior  mean 

P(t)     =  experimental  mean 

E''m(t))  =  posterior  mean 

v'      =  prior  variance 

V       =  experimental  variance 

v' '     =  posterior  variance 
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A 

Since  the  prior  of  p(t)  is  normal  and  the  distribution  of  p(t)  given  p(t)  is 
normal^  the  posterior  of  p(t)  is  normal  and,  as  can  be  shown  (see  l 191    p.  294-5), 
has  mean  and  variance 

1/v'      „,    r   .   .\  1/v 


^'•(P(^>^  -  lA  i/v  ^'  (P(^))    ^  i/.l  1/v  P(^> 


(6.18) 


v"    V   v' 

The  process  generating  p(t)  has  been  specified  in  (6.6): 

P(t)  =  kp(t-l)  +  (l-k)p°  +fep(t)  (6.6) 

As  of  the  beginning  of  t,  we  have 

E'(p(t))  =  k  E"(p(t-1))  +(l-k)p°  (6.19) 

To  simplify  notation,  let 

_  jTv] (6.20) 

^  "  1/v  +  1/v' 

Then,  substitution  of  (6.18)  into  (6.19)  yields  as  the  prior  mean 

E-  Cp(t))  =  k  a  E'(p(t-1))+  k(l-a)'^(t-l)  +  (l-k)p°  (6.21) 

If  k  is  near  one,  a  sufficiently  good  approximation  may  perhaps  be 

obtained  by  setting  k  =  1  in  the  above  expression.   This  has  the  advantage  of 

o 
eliminating  k  and  p  as  parameters  in  the  decision  rule  and  so  we  shall  do  it. 

However,  there  is  no  fundamental  difficulty  in  carrying  along  k  and  p   .   In 

that  case  (6.14)  would  be  replaced  by  (6.21). 

Under  the  k  =  1  approximation,  (6.21)  reduces  to  (6.14),  thereby 
justifying  an  expression  that  was  simply  postulated  in  Section  6.4: 

E'^P(t))  =  a  E'(^p(t-1))  +  (l-a)'^(t-l),  (6.14) 

In  addition,  we  now  have  a  value  for  a,  or  rather,  we  will  have  as  soon  as  we 
find  an  expression  for  v'  in  terms  of  the  given  parameters  of  the  model. 

To  do  this,  take  the  variance  of  each  side  of  (6.6).   Using  '  to  denote 
prior  and  "  posterior,  we  obtain 
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V(p(t))=  k  V"(p(t-1))  +  a 


Using  the  k  =  1  approximation^  the  above  plus  (6.18)  gives 
2 


v"    V    v' 

solving,      V  4.;  [1  +  (l  +  Wa/]  U   .  (6.22) 

2 
Since  _  1  0_  (6.23) 

^  ~  n  z\2 

we  can  now  set  a  by  (6.20)  . 

6.6   Choosing  x  (t) 


Profit  rate  is   a  random  variable  because  s(t)  is: 

p(t)  =  m  s(t)  -  x(t)  -  c 

s(t)  =  a(t)  +  p(t)x(t)-^  x^(t) 
Since  we  cannot  maximize  true  profit  we  choose  to  maximize  expected  profit. 
Suppressing  t  for  the  moment, 

E'(p)  =  m  [j;'(a)  +  E'(p)x-yx^J-  x  -  c  . 
This  is  maximized  by  choosing  x(t)  to  be 

X  (t)  =  ^'^P<f)  -  "■" 
o  2  IT 

This  justifies  (6.15)  of  Section  6. 

At  this  point  we  have  justified  the  decision  rules  of  Section  6  for  our 

specific  sales  response  model,  at  least  to  the  extent  of  the  k  =  1  approximation. 

It  remains  to  pick  the  parameters  n  and^a^of  the  experiment. 

6. 7  Designing  the  Experiment 

The  experimental  parameters  will  be  picked  to  minimize  the  sum  of  two 
losses:   the  loss  because  we  do  not  know  3(t)  exactly  and  the  loss  we  incur  trying 
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to  learn  p(t)  better. 

With  perfect  information  we  would  choose;     Instead  we  are  choosing-. 


x*(t) 


i(t)  -  1/m 
28* 


x^(t) 


E'(p(t))-  1/n 

2^r 


The  loss  compared  to  perfect  information  is  seen  from  (6.4)  to  be 

jl  =  m^  |x^(t)  -  x*(t)]  ^ 

Notice  above  that 

E'(xVc(t))   =  x^(t) 

;.   E'(i6  =  mirv'(x*(t)) 

=  ra^ 


^'0-r.^' 


(6.24) 


This  is  the  expected  loss  rate  relative  to  perfect  information  when  we  use  x  (t) 

In  the  2n  markets  used  for  the  experiment  the  expected  loss  rate  is  higher 
because  the  promotion  rate  is  deliberately  set  to  be  different  from  the 
best  available  value^  x  (t) .   Suppressing  t  for  the  moment^  the  experimental 
promotion  rates  are 

X,  =  x  -A/2 

1  o 

x^  =  X  +A/2 

2  o 

Consider  a  market  at  x   .   Its  loss  rate  compared  to  perfect  information  is 


^1  =  n[;i  --*J 
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2 
Therefore  E'(/)    =   mj-V(x*)    +  m2f^/4    . 

tn        .  .    2  , , 

The  same  expected  loss  rate  will  occur  for  x, 
of  the  experimental  deviation  is 

I  =mffJ/4  (6.25) 

The  total  expected  loss  rate  can  now  be  computed.   Let 
N  =  total  number  of  markets  in  the  country 
2n  =  the  number  of  experimental  markets 
P  =  the  average  number  of  households  in  a  market 
L  =  total  expected  loss  (dol/yr) 

L  =  N  P  ^  v'  -I-  2  P  m  5nA  (6.26) 

The  experimental  design  parameters  n  and  A  are  to  be  picked  to  minimize 

L.   First  we  observe  from  (6.26)  and  (6.22)  that  n  and  A    always  appear 

2  2 

in  L  in  the  combination  nA  •   We  shall  find  n^  to  minimize  L.   Then  we 

2 
can  more  or  less  trade  off  n  against  A   any  way  we  wish  as  long  as  nA     is  kept 


be  non-negative  and  2n  not  greater  than  N^  to  cite  two  extremes)  but  the 

flexibility  implied  is  interesting  and  desirable. 

2 
Rather  than  minimize  L  with  respect  to  nA  ;  we  minimize  with  respect 

to  the  dimensionless  quantity 
2a' 


2  2     2 

from  which  n^  can  immediately  be  calculated.   Substituting  n4  =  2a  /v  in  L, 

we  obtain 
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L  =  NP-ap  ^1+ 


+  ?  mg  a   /v 


Setting  d  L/dv  =  0  leads  to  an  expression  wiiich  optimal  v  must  satisfy: 
^_  2^ri+i-^~|^  (6.27) 


Although  (6.27)  contains  v  on  both  sides,  it  can  frequently  be  solved 
iteratively  by  setting  v^  =  0  and  recursively  calculating 

4v 


2ifa 


"  ^^f 


1+ 2- 


for  a  few  n  until  v^  converges  to  v  . 

6.8  A  Numerical  Example 

To  illustrate  the  behavior  of  the  adaptive  control  system,  we  construct 
a  numerical  example  of  the  model  assumed,  determine  the  optimal  experiment 
and  simulate  the  operation  of  the  system  under  the  proposed  decision  rule. 

i 

In  this  particular  model  the  data  assumed  available  to  the  company  are; 

y  =  100  =  curvature  of  sales  response  curve  (dol/hh.yr.) 

0=1=  period  to  period  standard  deviation  of  p(t)  (dimensionless) 

a  =  .016  =  standard  deviation  of  sales  rate  for  an  individual 
market  (dol/hh.yr.) 

N  =  1000  =  number  of  market  areas  in  the  country 

m  =  1/3  =  gross  margin  (dimensionless) 

For  perspective  we  note  that  national  sales  are  in  the  neighborhood  of 

10  million  dollars/year  or  .2  dol/hh.yr.  and  that  national  promotion  rate  is 

on  the  order  of  1  million  dollars/yr.  or  .02  dol/hh.yr.   Thus  a   implies  a 

coefficient  of  variation  for  an  experimental  unit  of  about  87o. 


There  is  little  conceptual  difficulty  in  setting  up  a  model  in  which  more 
parameters  are  measured  (e.g.  f   and  o  ),    but  the  computations  would  be  more  complex. 
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To  design  the  experiment,  the  appropriate  data  above  are  substituted 
into  (6.27)  which  can  be  solved  to  give 
V  =  .111, 

or  n^  =  2a  /v  =  48.0  x  lO'   (dol/hh.yr.)   . 
If  we  choose  A  so  that  the  high  and  low  spending  rates  are  25%  above  and 
below  the  national  rate  when  the  latter  is  .02,  we  obtain 

A  =  .01  dol/hh.yr. 
Then 

n  =  48  markets. 

This  completes  the  experimental  design:   In  each  time  period  48  markets  will 

be  run  at  a  rate  .005  dol/hh.yr.  above  the  national  rate  x  (t)  and  another 

o 

48  the  same  amount  below.   The  resulting  experimental  standard  error  of  p 

is  .333  ^=v^.   This  makes  the  experiment  a  fairly  precise  one,  especially 

when  compared  to  the  period  to  period  standard  error  of  p(t)  which  is  a.  =  !• 

P 

Since  we  are  predicting  future  p(t)  from  past  estimates  and  p  is  changing 
fairly  fast  (standard  error  =  1),  we  will  tend  to  rely  on  the  most  recent 
experiment  (standard  error=.333). 

In  order  to  operate  we  need  smoothing  constant  £.   First  v'  is  found 
from  (6.22)  to  be  1.10.   Then  (6.20)  gives 

a  =  .092 
and  (l-a)=.908,  so  that  we  do  indeed  put  most  reliance  on  the  most  recent 
experiment. 

The  value  of  x  (t)  at  each  t  can  be  calculated  from  (6.14)  and  (6.15)  or, 
combining  them^  from 

x^(t)  =  a  x^(t-l)  +  i^ 


Specifically, 

x  (t)  =  ( 


|f^(t-l)  -  1/ml. 
.092)  x^(t-l)  +  .00454|^(t-l)  -  3.o] 
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We  are  now  set  to  operate  the  company:   Given  a  starting  value  Xq(0)  and  a 

A 

sequence  of  experimental  results  P(t)^  we  can  generate  the  promotional 

rate  x  (t)  . 
o 

The  underlying  market  behavior  is  simulated  from  (6.6)  using  the 
specific  parameters 
k  =  .9 

s„  • '  • 

This  is  done  by  generating  a  sequence  of ^(t)  having  a  normal  distribution 

2 
of  mean  zero  and  variance  a^   =  1  and  substituting  them  into 
P 

p(t)  =  .9  p(t-l)  +  .7  +€(t). 
For  the  a(t)  process,  we  take  a(t)  =  .1  dol/hh.yr.   A  varying  of  a(t)  might 
be  more  realistic  but,  since  it  does  not  enter  into  any  decisions,  we  simply 
make  a(t)  constant. 

Given  P(t),  the  experimental  results  of  t  are  simulated  by 

p(t)  =  p(t)  +€^it)    . 

Here£'^(t)  is  a  random  normal  number  with  mean  zero  and  variance  v=.lll. 
B 

(We  could  simulate  each  of  the  2n  test  markets  separately  to  generate  P(t)_, 
but  it  is  obviously  easier  to  simulate  the  final  experiment  result  directly.) 

The  company  uses  the  p(t)  to  generate  x  (t) .   The  final  sales  outcome  can 
be  calculated  from  this,  p(t),  a,  and  y  : 

s(t)  =  .1  +  p(t)x  (t)  -  100  I  X  (t)l^  dol/hh.yr. 


t°i 


The  sales,  in  turn,  can  be  used  to  calculate  profit.   Our  principal  criterion, 
however,  is  the  loss  relative  to  profit  we  could  make  using  perfect  information. 
A  side  calculation  gives  the  best  promotion  rate  under  perfect  information: 


Kt)-3 

200 
The  loss  rate  relative  to  this  is,  from  (6.4): 


X  (t)  =  ^-^^   dol/hh.yr. 


J^  =  33.3  x*(t)-x^(t)r  dol/hh.yy. 
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The  further  loss  resulting  from  the  experimental  deviations  has  not  been 
simulated,  but  its  expected  value  can  be  calculated  from  (6.25).   For 
one  market 

/^  =  8.33  X  10'^  dol/hh.yr. 

Although  most  quantities  fluctuate  with  time,  p(t)  does  have  a  long-run 
average:   p  =7.   If  this  were  always  the  value  of  p(t),  the  optimal  x(t) 
would  be  x  =.02  and  the  corresponding  s(t)  would  be  s  =,20.  Theae  values 
make  convenient  reference  points:   Losses  can  be  expressed  as  a  %   of  s°  and 
we  can  also  compare  performance  of  the  adaptive  system  with  a  system  that 
simply  sets  x(t)=x   . 

The  simulation  results  are  shown  in  Figure  6,1.  40  time  periods  are 

o  o 

shown.  (The  series  was  started  with  x(o)  =  x  and  p(o)  =  p  and  ran  for  10 

periods  before  the  present  data  was  taken.)   plotted  are  p(t),  which  is 

driving  the  system,  x  (t),  by  which  the  company  responds,  and  the  resulting 

s(t)  and  jP(t).  The  latter  is  expressed  as  a  "L   of  s  , 

We  see  that  the  response  of  the  adaptive  control  system  is  quite  good, 

although,  by  necessity,  changes  is  x(t)  lag  changes  in  p(t)  by  one  time 

period.   The  losses  are  generally  small,  although  occasional  peaks  occur 

where  p(t)  has  changed  substantially  and  x(t)  has  not  yet  caught  up.   The 

40  period  average  is: 

average  ^(t) 

.,      ,     o         =  .^73 
as  7o  of  s 

This  may  be  compared  with  the  expected  value  (under  the  k=l  appoximation) 

calculated  from  (6.24): 

i^  =  .457% 
o 
s 

The  extra  loss  for  experimental  deviations  in  the  2n  test  markets  as  a  °k 

of  national  sales  is 


u 


1_ 


tn  o 


0  0„W   B 


d 


x(t)(d/h  yr) 


s  ».  a   J3  S   g  ° 

s(t)(d/h  yr) 


M   M   1-4 


^(oa  of  8°) 
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— ^  =  .040%  . 
Ns° 

The  total  expected  loss  is  therefore  .4971  of  the  sales  s   . 

By  way  of  contrast,  the  average  loss  for  a  system  using  a  constant 

x(t)  =  x°  =  ,02  over  the  40  periods  of  Figure  6.1  is  .854?,  of  s° 

demonstrating  the  value  of  estimating  and  following  p(t)  in  the  particular 

case  considered. 
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